Skip to content

End-to-end notebook template generation script#282

Merged
validbeck merged 38 commits intomainfrom
beck/sc-7899/move-templates-from-documentation-to-validmind
Jan 3, 2025
Merged

End-to-end notebook template generation script#282
validbeck merged 38 commits intomainfrom
beck/sc-7899/move-templates-from-documentation-to-validmind

Conversation

@validbeck
Copy link
Collaborator

@validbeck validbeck commented Jan 2, 2025 •
edited
Loading

Internal Notes for Reviewers

sc-7899

Migrated Jupyter Notebook templates

I moved the templates that used to live in documentation/internal/templates/jupyter-notebooks to validmind-library/notebooks/templates:

New templates folder Updated notebooks README
Screenshot 2025-01-02 at 1 13 20 PM Screenshot 2025-01-02 at 1 13 53 PM

Interactive end-to-end notebook template generation

There is now a fancy new Python script and an accompanying notebook:

  • notebooks/templates/e2e_template.py
  • notebooks/templates/e2e-notebook.ipynb

You can either run the notebook, or call make notebook from the root of the validmind-library repo:

validmind-library/Makefile

Line 83 in 17bc771

notebook:

The functionality of the script and the notebook is the same:

  • It asks the user for the filename of the new Jupyter Notebook, and creates a the file in notebooks/code_sharing
  • It asks the user for the title of the notebook and creates a markdown cell at the top of the notebook, applying the input as an h1 header
  • It asks the user whether or not they want to append the following mini-templates to the notebook:
    • notebooks/templates/about-validmind.ipynb
    • notebooks/templates/install-initialize-validmind.ipynb
    • notebooks/templates/next-steps.ipynb
    • notebooks/templates/upgrade-validmind.ipynb
  • Shows you the changed files you need to commit outside of notebooks/templates

Install & initialize ValidMind

If the user chooses to append the contents of this template to the notebook, they will also be prompted to enter:

  • A {template} name
  • A {use-case}
Default Example generated notebook
Screenshot 2025-01-02 at 1 24 57 PM Screenshot 2025-01-02 at 1 25 32 PM

Cell IDs

The templates have been broken down into individual cells with cell IDs, in preparation for maybe being able to bulk-edit notebooks in sc-4132. There are some other considerations, like how all pre-existing notebooks will need to be recreated with this templating script/notebook, and how we can work around injected table-of-contents anchors, but, hey, it's a head start.

Template cell ID Generated cell ID
Screenshot 2025-01-02 at 1 24 48 PM Screenshot 2025-01-02 at 1 25 47 PM

@validbeck
Stole templates from documentation repo
e22acaa
@validbeck
Edits to templates
75f3f7d
@validbeck
Updating README
bad8488
@validbeck
Testing a script
e5a68ee
@validbeck
Adding title
b8bc4de
@validbeck
Setup the other cells
0509213
@validbeck
Test append
4ddf5fd
@validbeck
Test adding IDs to cells
cb9fb8b
@validbeck
Add install info
e2f9bfe
@validbeck
Add next steps & upgrade info
ee110fd
@validbeck
Cleanup & testing Makefile
69b0cdf
@validbeck
Draft for notebook wording
53c81f5
@validbeck
README
455b867
@validbeck
ToC
1d78229
@validbeck
Wording tweaks
ae96d19
@validbeck
Updated main notebooks/ README
39ddc2a
@validbeck
Modifying Makefile
476a50a
@validbeck
Tweaking the READMEs
09ca0d8
@validbeck
Trying somtehing with the cell IDs
981201b
@validbeck
It works, let's try it again
689a46f
@validbeck
Spiltting the templates into more cells
cb92bbc
@validbeck
Hm
61538a7
@validbeck
Asking for template & use-case
00313a7
@validbeck
Whoo
56bf981
@validbeck
Typo
fec89f7
@validbeck
Tweak
8bbb373
@validbeck
Almost theeeere
5c2d0dd
@validbeck
Final? tweak
5cf43cc
@validbeck
I lied
8cf151e
@validbeck
Wordiiiing
2254714
@validbeck validbeck added the internal Not to be externalized in the release notes label Jan 2, 2025
@validbeck validbeck self-assigned this Jan 2, 2025
@validbeck validbeck requested review from cachafla and nrichers January 2, 2025 21:36
cachafla
cachafla approved these changes Jan 3, 2025
View reviewed changes
Copy link
Contributor

@cachafla cachafla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very fancy 😄 great job!

One minor suggestion for the future (no need to make any changes now) is to leverage @click for building the CLI:

@github-actions
Copy link
Contributor

github-actions bot commented Jan 3, 2025

PR Summary

This pull request introduces several enhancements to the ValidMind project:

  1. Makefile Update: A new notebook command has been added to the Makefile. This command runs a Python script located at notebooks/templates/e2e_template.py and filters the output of git status to exclude changes in the notebooks/templates directory.

  2. Notebook Templates: Several Jupyter notebook templates have been added under the notebooks/templates directory. These templates include:

    • about-validmind.ipynb: Provides an overview of ValidMind, its key concepts, and prerequisites.
    • e2e-notebook.ipynb: A template for generating end-to-end notebooks for ValidMind.
    • install-initialize-validmind.ipynb: Instructions for installing and initializing the ValidMind Library.
    • next-steps.ipynb: Guidance on next steps after using the ValidMind Library.
    • upgrade-validmind.ipynb: Instructions for upgrading the ValidMind Library.

  3. Template README: A new README file has been added to the notebooks/templates directory, providing a quick start guide for creating code samples using the provided templates.

  4. Python Script for Notebook Generation: A Python script e2e_template.py has been added to facilitate the creation of new Jupyter notebooks by combining various mini-templates. This script includes functions for creating notebooks, setting titles, and appending content from mini-templates.

  5. Documentation Update: The notebooks/README.md file has been updated to remove the deprecated notebooks/archive section and add a new section on contributing code samples.

Test Suggestions

  • Test the new make notebook command to ensure it correctly runs the Python script and filters the git status output.
  • Verify that each Jupyter notebook template can be opened and executed without errors.
  • Check that the Python script e2e_template.py correctly creates a new notebook and appends content from mini-templates.
  • Ensure that the README files provide accurate and clear instructions for using the templates and contributing code samples.
  • Test the integration of the new templates with the ValidMind Library to ensure they function as expected.

@validbeck validbeck merged commit f56f23c into main Jan 3, 2025
6 checks passed
@validbeck validbeck deleted the beck/sc-7899/move-templates-from-documentation-to-validmind branch January 6, 2025 15:31
@validbeck validbeck mentioned this pull request Jan 7, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cachafla cachafla cachafla approved these changes

@nrichers nrichers Awaiting requested review from nrichers

Assignees

@validbeck validbeck

Labels

internal Not to be externalized in the release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@validbeck @cachafla